Skip to content

Data generations: keep data from different sources apart - #127

Merged
MrScripty merged 3 commits into
mainfrom
feat/data-generations
Nov 20, 2026
Merged

MrScripty merged 3 commits into
mainfrom
feat/data-generations

Conversation

@MrScripty

Copy link
Copy Markdown
Owner

A dataset or step may declare the generation its data comes from (for example the solver version that produced it). Derived datasets inherit it, and check keeps generations apart:

  • inputs that disagree, or a declared generation that differs from its inputs, are an error;
  • a recipe that trains on two generations is an error, unless it declares "generations": "mixed", which makes it a warning;
  • a benchmark whose subjects span generations gets a warning unless it declares "mixed";
  • labelled data that takes in unlabelled data gets a warning naming the input.

Labels are opt-in: an unlabelled project checks and reads exactly as before. The Map and Data tabs show a generation chip and a Generation filter.

Spec: docs/plans/training-and-curation/spec-generations.md. Tests: project suite 921 OK, agent tests 43 OK (4 skipped), the new browser check (7) and the workspace check (24) pass.

🤖 Generated with Claude Code

MrScripty and others added 2 commits November 19, 2026 16:14
…models

A dataset or step may declare the generation its data comes from; derived
datasets inherit it, inputs that disagree are an error, a recipe mixing
generations is an error unless it declares "generations": "mixed", and a
benchmark whose subjects span generations warns. The map, definition and
definitions reads carry the generation where there is one; the Map and the
Data tab show it as a chip, with a Generation filter on the Map.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dataset, step, recipe or benchmark that has a data generation (declared,
inherited or mixed) and takes a dataset or step without one is warned at that
input, so unlabelled data cannot be mixed into a generation unnoticed. An
input left without a generation by its own conflict is not named; projects
that label nothing are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: b39db45c-6f89-42f2-9cdb-079e020e36ac
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Trials as long as the budget could finish just before it ran out on a slow
runner, leaving no stopped trial; each trial is now far longer than the budget.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MrScripty
MrScripty merged commit 958c343 into main Nov 20, 2026
3 checks passed
@MrScripty
MrScripty deleted the feat/data-generations branch November 20, 2026 02:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant